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Abstract 

This is a didactic text (CDT-1) discussing the interrelationship between the concepts of matrix product, complex 
networks, and diffusion. The matrix product is interpreted as implementing several independent scalar products 
with the same set of input variables. This principle is then adopted to develop several relationships between the 
considered concepts. 


“Tessono con i fili una figura simile che vorrebbero phi 
complicata e insieme piu regolare dellaltra.” 

Le Citta Invisibili, I. Calvino 


1 Introduction 

Matrix products, complex networks, and diffusion. 
The former concept derives mainly from linear algebra 
(e.g. [1]). Complex networks, or graphs, are an impor¬ 
tant concept that has been extensively investigated re¬ 
cently (e.g. [2, 3]). Diffusion is a dynamical process 
underlying many of natural phenomena, such as tem¬ 
perature changes, being typically studied in statistical 
physics (e.g. [4]). Three different concepts, deriving from 
distinct respective areas. Would there be any relation¬ 
ship between them? 



Figure 1: What are the relationships between matrix product, complex 
networks, and diffusion? Can such interrelationships promote a better 
understanding of these concepts? 


Though at a first glance these concepts may not seem 
to have much in common, as we will show in this text 1 , 
they are closely interrelated. The key for linking these 
concepts is the operation of linear combination of several 
variables, another concept derived from linear algebra. 

After presenting the concept of linear combination, 
and how it is related to the scalar product , it will be 
shown that the product between a matrix A and a vec¬ 
tor x, yielding a new vector y , can be understood as 
several independent scalar products respectively imple¬ 
mented between each of the rows of A and the same 
imput vector x. These linear combinations define a lin¬ 
ear transformation mapping from the input vector x to 
the output vector y. 

Then, by making y = x, i.e. by placing the result of 
the product A x into x, it will be shown that the matrix 
product performs a redistribution of the original values 
in x among its own elements, each of them receiving a 
linear combinations of the values in the others elements 
(possilblt also including itself). We will also see that this 
distribution is governed by a graph or complex network 
among the elements of x, with the links being deter¬ 
mined by matrix A , now called the weight/strength or 
adjacency matrix. Next, we will show how to easily iden¬ 
tify the output and input of each element of x, which are 
henceforth called the respective input and output star , 
while the number of input and output links of an el- 
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ement of ox correspond to its respective indegree and 
outdegree. 

We will also discuss that, by repeating the distribution 
of the values in x and using weights that conserve the 
overall mass, a dynamics is obtained that corresponds 
to diffusion of quantities such as mass or energy. 

The conceptual framework reported in this work pro¬ 
vides a potentially powerful way to complement, in joint 
fashion, our understanding of matrix products (as being 
related to graps), complex networks (in terms of ma¬ 
trix products), as well as diffusion (in terms of repeated 
matrix products). 

2 Linear Combinations and the 
Scalar Product 

Figure 2 illustrates the concept of linear combination of 
4 given variables aq, x 2 , ^3, and x4 into a new variable y. 
A new quantity (variable y) is defined by adding specific 
quantities of the input variables defined by respective 
weights <21, <22, <23, and <24, i.e.: 

4 

y = a\x 1 4- a 2 x 2 + a 3 x 3 + a 4 x 4 = ^ tqaq (1) 

i=t 



proportions. Yet another example: sounds are composed 
by the linear combination of basic sinusoidal functions, 
with respective weights (the Fourier series [ 5 ]). 

For generality’s sake, we can have N input variables 
and weights, with the latter typically assuming real val¬ 
ues, but complex weights are also employed. It is in¬ 
teresting to observe that some of the weights ai can be 
equal to zero, implying that the respective variables will 
effectively not take part in the linear combination, as 
illustrated in Figure 3 . 



Figure 3 : An example of linear combination where two of the weights, 
namely a± and 0,3, are equal to zero. In this case, we have y = a\X\ + 
CL2X2 + <232:3 + 0,42:4 = CL2X2 + 042:4. 

Linear combinations can be more compact and conve¬ 
niently expressed in terms of scalar products , i.e. 

N 

y = (a, x) = (x, a) = ^2 a i x i ( 2 ) 

i=1 

The scalar product is sometimes also called dot prod¬ 
uct or inner product. It can also be expressed in ma¬ 
trix form, namely as the product between the weight 
row vector a = [ai, <22,..., a at] and the column vector 
x = [aq, x 2 ,..., xjsi] 1 (T stands for the transpose of a 
vector or matrix), i.e.: 


Figure 2: Graphical representation of the linear combination of 4 vari¬ 
ables 2?i, X2, X3, and 2:4. Each of these variables is multiplied (repre¬ 
sented as *) by a respective weight = a 1, 0,2, <2,3, and 04 and then 
added together to yield the output variable y. 


There are several real-world situations directly related 
to the concept of linear combination. For instance, we 
have a cake recipe, where the cake (y) is obtained by 
adding up several ingridients {xf) in respective quantities 
(ai). Similarly, a specific paint color can be obtained 
by adding together paints of several colors, in respective 


y - (a, x) — ax — a\ a 2 a 3 a 4 


Xi 

X2 

X3 


( 3 ) 


3 Matrix Products 

Consider the four linear combinations shown in Ta¬ 
ble l(a-d). Though the 4 input variables, namely aq, 
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X2 , £3, and x 4 , are the same for all these four cases, the 
respective weights may vary in each case, being there¬ 
fore represented with the help of an additional index, so 
that the weights now have the general form a^j. As the 
weights are not necessarily the same in the four cases, 
the output variables are very likely distinct, being rep¬ 
resented as yi = 2/1,2/2,2/3, and y A . 

As discussed in the previous section, we know that 
each of these four linear combinations of x\, X2, X3, and 
X4 can be expressed by respective scalar products, i.e.: 

y% = d\,\X\ + a^ 2 x 2 + ai ? 3 x 3 + a 1 A x 4 = (a[,x) (4) 

y 2 = a 2 ,ixi + a 2 , 2^2 + 02 , 3 X 3 + o 2 , 4^4 = ( 02 , x) (5) 

2/3 = a 3 ,ixi + a 3;2 x 2 + a 3?3 x 3 + a 3?4 x 4 = (a£, f) (6) 

2/4 = ^4,1^1 + «4,2^2 + «4,3^3 + ^4,4^4 = ( 04 , x) (7) 

where dj = [a M a i;2 a i?3 a i?4 ] T . 

These four equations correspond directly to the matrix 
product between a matrix A = [i a^j\ and x, i.e. 


ll 

55 


a 1,1 a l,2 0*1,3 ^1,4 


" Xi 

y 2 = (a 2 ,x) 


02,1 02,2 02,3 a 2,4 



2/3 = (a3,x) 


03,1 03,2 03,3 03,4 


%3 

. Va = {a a, x) . 


_ a 4? i a 4?2 a 4?3 a 4?4 _ 


X 4 


( 8 ) 

or, in more summarized fashion: 

y = A x ( 9 ) 

Equation 8, or its more synthetic form given in Equa¬ 
tion 9 , implements all the four linear combinations in 
Figure 1 in ‘parallel’ fashion, in the sense of them being 
performed at once by the matrix formulation. It should 
be remembered that the real computer execution of the 
scalar products corresponding to the linear combinations 
will depend on the adopted hardware in which the ma¬ 
trix product is performed, which can be sequential or 
vector/parallel. 

Irrespectively of the way in which the matrix product 
is executed, it certainly provides a compact and elegant 
way to summarize the several involved scalar products, 
which maps the original vetor x into the new vector y. 
As such, the matrix product can be understood as a 
transformation. More specifically, it is shown in linear 
algebra that there is a complete correspondence (isomor¬ 
phism) between matrix products and linear transforma¬ 
tions (e.g. [1]). 

Examples of linear transformations include the 
Hadamard and Fourier transform, principal component 
analysis, and geometrical rotations [ 5 ]. At the same 


time, the matrix product can also be understood to pro¬ 
vide a map from the states of a given complex networks, 
having the states of its nodes in correspondence with the 
input variables Xi, into the states associated to the nodes 
yi of another complex network. In this way, the matrix A 
defines how the nodes of the input complex network are 
mapped into the nodes of the second, coupled network. 

This construction is particularly interesting for neu¬ 
ronal networks (e.g. [6]), in which case the matrix A 
specifies the way in which the output of the neurons 
of a subsequent layer x are mapped into a subsequent 
layer y. These relationships between the matrix prod¬ 
uct and Fourier transforms and neuronal networks are 
particularly broad and interesting and may be covered 
in eventual future didactic texts (CDTs). 

As we shall see in the next section, the matrix product 
can also be associated to the interconnections within a 
single complex network, therefore assuming the role of 
the adjacency matrix in case of A contains only Os and 
Is, or strength matrix in the more general case of the 
elements of A being real values. 

4 Complex Networks 

We have already seen how the matrix product y = Ax 
can be understood as a transformation from the states 
of an input network, into an output network. A partic¬ 
ularly interesting situation arises when we feed back the 
output into the own input of the matrix product, i.e.: 

x t+1 =Ax t (10) 

Observe that, since the input and output variables are 
the same, we need to introduce a temporal order in terms 
of time steps t = 0 , 1 , 2,..., N t . For the case of N = 4 , 
the respective matrix formulation becomes: 
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03,1 

_ 04,1 

03,2 033 a 3;4 

04,2 04,3 04,4 _ 
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So, we have that the current values x\ of the variables 
Xi, which will henceforth be called the state of the sys¬ 
tem, are intermixed by each of the linear combinations 
in the above matrix product, and the results are updated 
into the respective states. As such, matrix A specifies 
the way in which the states of the variables Xi are redis¬ 
tributed among themselves. This redistribution is per¬ 
formed through a network of interconnections between 
the elements, or nodes jq. Thus, matrix A, together 
with the nodes associated to Xi, can be understood as 
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Table 1 : Four potentially different linear transformations of the same input variables x\, X2 , X3, and X4, producing y 1, y 3 and 7/4, respectively. 
The four scalar products implementing these linear combinations can be effectively summarized by the matrix product y = Ax. 
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a complex network. When iterated, the matrix product 
gives rise to a dynamic system. 

Let’s investigate in more depth the interrelationship 
between the matrix product and complex networks. 
Consider the network (or graph) depicted in Figure 4. 



Figure 4 : A network, or graph, containing N = 7 nodes, each of which 
having Xi as a respective state variable. The interconnection between 
these nodes, as implied by the matrix product x t+1 = A x l are 
completely specified by the matrix A. Hence, a complex network is 
completely specified by A and Xi. 


As it will soon be seen, the above complex network 
exaple is defined by the respective matrix A given as: 


A = 


0 0 10 10 1 
1 0 0 0 1 0 0 
0 0 0 0 1 0 1 
0 10 0 110 
1 0 0 0 0 1 0 
0 0 0 0 0 0 0 
0 0 0 1 0 0 0 


(13) 


The set of input edges of a node i is henceforth called 
its instar of i. Figure 5 illustrates (highlighted) the in¬ 
star of node i = 5 in the previous network. The number 
of elements in ini corresponds to the indegree of node i. 
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Observe that though the considered matrix A con¬ 
tains only 0s and Is as entries, other real values are also 
possible. The former type of matrices is known as adja¬ 
cency matrices , while the latter is often called strenght 
or weight matrices. 

The graph in Figure 4 has N = 7 nodes, with re¬ 
spective states Xi following the recurrence Equation 10. 
Now, we have already seen that each new state x* +1 
is obtained in terms of the respectively associated scalar 
product, i.e. x * +1 = a^x\ +^, 2^2 “K • - + i x 7 ( see Fig¬ 
ure 1). It immediately follows that the input of the new 
state x* +1 can be obtained directly from the respective 
row i of matrix A. For instance, the input of node 5 can 
be easily obtained from A by taking the non-null indices 
of its fifth row as corresponding to the set in§ = {1,6}, 
i.e.: 


Figure 5 : The instar of the node i = 5 of the network in Figure 4 . 
The indegree of this node, therefore, is equal to 2 . 

Similarly, we can also be interested in obtaining all 
the outgoing connections emanating from a node j, as 
implied by the interconnecting matrix A. Going back 
to Table 1, we can easily observe, by comparing the 
four scalar products (a-d), that a same input node i 
will be multiplied by the j— th element of the respective 
weight vectors. This position corresponds to for 
i = 1, 2,..., 7. So, the outgoing connections of a given 
node j can immediately be obtained as corresponding 
to the non-null entries in the column j of matrix A. In 
the case of the previous example, for j = 5, we therefore 
have the outgoing set of links out$ = {1, 2, 3,4}, i.e.: 
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Figure 6 shows the outstar of node j = 5 for the graph 
in Figure 4. This node j = 5 has outdegree equal to 4. 



Figure 6 : The outstar of the node i = 5 of the network in Figure 4 . 
The outdegree of this node, therefore, is equal to 4 . 

So, we have seen that while the rows of A specify the 
input to each of the nodes i, the columns of that same 
matrix provide the output of each node. We thus have 
that the matrix A completely specifies the interconnec¬ 
tions between any node i, so that A, together with the 
elements Xi, define a respective complex network, as im¬ 
plied by the matrix product Ax 1 . 

5 Diffusion 

We have already verified that the matrix product x t+1 = 
Ax 1 implements a redistribution of the current values 
(states) xi at each of the nodes i. In case the weights 
ctij assume generic real values, each of the current state 
value x\ can be amplified or reduced in case the mag¬ 
nitude of the respective weight is larger or smaller than 
one. 

A particularly interesting situation arises when the 
value of the current stat is conserved. Indeed, if we in- 
terprete these values as corresponding to some sort of 
‘mass’ or ‘energy’ being circulated through the network, 
it is reasonable to consider the situation where this mass 
or energy is conserved. In order to do so, we take into 
account the outstar of a node j. The matrix product 
implies that the value Xj at each node i is sent to all 
the other nodes i with respective weights eqj, so that 
each of the outgoing node i receives a quantity ctijXj. 
The conservation of the quantity x l - at a node j requires 
that the sum of new quantities to be replaced into the 
outgoing nodes, i.e. cti,jXj , be equal to Xp i.e.: 

N 

Xj = a i,j x j ( 15 ) 

i= 1 


by dividing both sides by Xj, we have: 

N 

i = XA,j ( i6 ) 

implying that mass/energy conservation requires that 
the sum of all columns of A are equal to 1. Because 
the quantities cannot be destroyed, we also have that all 
weights ctij need to be larger or equal to 0. Matrices A 
that satisfy these two properties are said to be stochastic 
(e.g. [7, 4]). In this case, the normalized elements of A 
can be treated as probabilities. An example of such a 
matrix is: 
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When A is a stochastic matrix, the iteration of the 
product = Ax 1 can be verified to implement dif¬ 
fusion of the quantities at each of the network nodes. 
Though a more formal proof requires further formalities 
and calculations, the above reasoning provides a concep¬ 
tual framework for better understanding, in an intuitive 
way, the relationship between matrix products and dif¬ 
fusion dynamics. There are also interesting relationships 
between the concepts developed above and the area of 
Markov chains of probability (e.g. [4, 7]). 

6 Concluding Remarks 

Matrix products, complex networks, and diffusion are 
not always strongly associated one another. Deriving 
from distinct areas, these concepts are, each on its own 
ways, very important in science and technology. In the 
present work, we have shown that these three concepts 
are, indeed, much more interrelated than it could ini¬ 
tially appear. In summary, matrix products implement 
‘in parallel’ several linear combinations of the input vari¬ 
ables, giving rise to new output variables. This concept, 
illustrated in Table 1 provides the key to the interrela¬ 
tionships discussed in the current work. 

Through the linear combination concept, the matrix 
product has been shown to correspond to linear transfor¬ 
mations of the original variables being, therefore, intrin¬ 
sically related to important transforms such as Fourier 
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and principal component analysis, as well as to the in¬ 
terconnection between successive layers in neuronal net¬ 
works. When the output variables are used to replace 
the current state of the input variables, we have a par¬ 
ticularly interesting situation that it has been intuitively 
shown to correspond to a dynamics of redistribution of 
the state values among the nodes of a complex network 
whose nodes correspond to the input variables and the 
interconnections are specified by the matrix A. We have 
also argued conceptually that when the elements of A are 
normalizes so that each of its lines adds up to one (defin¬ 
ing a stochastic matrix), the matrix product x t+1 = Ax 1 
implements diffusion with conservation of the state val¬ 
ues. 


It is believe that the reported insights and interre¬ 
lationships can not only promote a better understand¬ 
ing not only of the concepts of matrix product, com¬ 
plex networks, and diffusion taken separately, but also 
contribute to a better appreciation of the important 
intrinsic relationships between these concepts. There 
are additional important such interrelations - such as 
those underlying the Fourier transform, eigenstates, dif¬ 
fusion, stochastic processes, and hierarchical organiza¬ 
tion of complex networks, and these are planned to be 
covered in future CDTs. 
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Costa’s Didactic Texts - CDTs 


This is a Costa’s Didactic Text (CDT). CDTs 
intend to be a mid-term between a formal scien¬ 
tific article and a dissemination text in the sense 
that they: (i) explain and illustrate concepts in a 
more informal, graphical and accessible way than 
the average scientific article; and, at the same 
time, (ii) provide more in-depth mathematical 
developments than a typical dissemination work. 

It is hoped that CDTs can also provide new 
insights and analogies concerning the reported 
concepts and methods. We hope these character¬ 
istics will contribute to making CDTs interesting 
both to beginners as well as to more senior 
researchers. 

Each CDT focuses on a few interrelated con¬ 
cepts. Though attempting to be relatively self- 
contained, CDTs tend to be shorter than the av¬ 
erage scholar article. Links to related material 
— which, with time, may include other CDTs — 
are provided in order to complement the covered 
subjects. 
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